Run coding_tools_env verification outside the notebook kernel - #1580

Summary

Ports #1233 to coding_tools_env. Verify commands, the reward file clear and the reward file read went through the code-interpreter kernel (run_shell sends a subprocess.run cell, read_file a Path.read_text cell), so their results depended on kernel state. They now run as their own sandbox processes through E2B's process API, the way jupyter_env does since #1233.

Refs #1232

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation
• New environment
• Refactoring

Alignment Checklist

Before submitting, verify:

• I have read .claude/docs/PRINCIPLES.md and this PR aligns with our principles
• I have checked .claude/docs/INVARIANTS.md and no invariants are violated
• I have run /pre-submit-pr (or bash .claude/hooks/lint.sh and tests) and addressed all issues

RFC Status

• Not required (bug fix, docs, minor refactoring)
• RFC exists: #___
• RFC needed (will create before merge)

Details

• E2BSandbox.run_command is the jupyter_env one with the same rules: it starts the command with background=True as root in /home/user and waits on the handle. A non-zero exit (CommandExitException, matched by its exit_code) is a failed result. A command that cannot be started, or a wait that ends without an exit status, is raised after a best effort kill, so it is never scored.
• It returns a ToolResult in the shape run_shell uses here (stdout and stderr joined, error="exit_code=N"), so verify_results look the same as before. Its default timeout is 30 s like run_shell; verify commands still pass 120 s.
• _run_verify_commands uses it for the mkdir, the clear and each verify command, and _read_reward_override reads the file with cat through it instead of read_file.
• The agent's bash and file tools and the setup commands are unchanged, so nothing the agent sees changes.

Where this differs from jupyter_env:

• The agent here has no cell tool, so the kernel is not something it writes to directly. Its shell does run as root next to the kernel, though, and verification shouldn't depend on kernel state either way.
• run_shell here already reported real exit codes, so honest scoring doesn't change.
• One behaviour change to look at: a verify command that runs past its 120 s timeout used to count as a failed command (the cell raised TimeoutExpired). Now, as in jupyter_env, it is raised: no reward, and the episode doesn't finish. A hung test run is common in coding tasks, so say if you'd rather keep counting it as a failure here.

Same limit as #1233: this is not an isolation boundary. E2B starts commands as bash -l -c, so a bash call that writes an EXIT trap into /root/.bash_profile makes a failing verify command pass. A test pins that, and the README says so. On main the same root shell can already rewrite the tests, a conftest.py or /bin/sh, so it isn't a new class of problem; #1232 tracks the fix.

Test Plan

New tests in tests/envs/test_coding_tools_environment.py drive the real E2BSandbox against a stand-in kernel (cells run with a namespace that persists) and a stand-in process API (each command a fresh bash -l -c with a real exit status), with /home/user and /root mapped to temp dirs:

• jupyter_env: verification runs in the agent's own kernel, so a rollout can report any reward #1210's cell rebinding subprocess.run reaches the kernel: both exit 1 verify commands still fail and the reward is 0.0. No verify command reaches the kernel.
• A kernel cell forges Path.read_text to return "1.0"; a verify command that writes 0.8 yields 0.8.
• A kernel cell calls os.chdir; pwd in verification still reports the notebook dir. Every call goes out in the background, as root, in /home/user, and the verify command keeps its 120 s timeout.
• run_command unit tests, as in Run jupyter_env verification outside the notebook kernel #1233: non-zero exit, wait failure plus kill (also when the kill returns False or raises), start failure, and e2b.CommandExitException itself.
• A sandbox that fails during verification gives a tool error, no verify results, no reward and done false.
• The bash tool writes a poisoned /root/.bash_profile and a failing verify command passes (the limit above).
• The existing submit test also asserts verify went through run_command, not run_shell; the reward file fake now answers both.

PYTHONPATH=src:envs pytest tests/envs/test_coding_tools_environment.py, with e2b 2.21.0 and e2b-code-interpreter 2.6.2 as locked in envs/coding_tools_env/uv.lock:

• main with only the new tests: the 11 new or changed cases fail, the other 13 pass.
• this branch: 25 passed. tests/envs/test_jupyter_environment.py still passes.
• One more test passes on main too and guards this port: the reward file is parsed from stdout alone, so a login profile that prints to stderr cannot turn a valid override into a fallback to the pass rate.

usort check, ruff format --check and ruff check are clean on the test file, and ruff check on the env files. The two env files were not ruff format clean on main; I left those lines alone, and the findings on them are the same as on main. scripts/sync_env_docs.py --check passes after refreshing the docs stub.

Not run against a live E2B sandbox. With an E2B_API_KEY, this checks the change and the user and directory assumption in one go:

from coding_tools_env.server.coding_tools_env_environment import CodingToolsEnvironment
from openenv.core.env_server.mcp_types import CallToolAction

env = CodingToolsEnvironment()
env.reset(verify=["exit 1", 'test "$(id -u)" -eq 0 && test "$(pwd)" = /home/user'])
# stands in for code that reaches the kernel by another route
env._sandbox._sbx.run_code(
    "import subprocess\nclass F: returncode = 0; stdout = ''; stderr = ''\n"
    "subprocess.run = lambda *a, **k: F()"
)
env.step(CallToolAction(tool_name="submit_solution", arguments={}))
print(env.state.last_reward, [r.ok for r in env.state.verify_results])
# this branch: 0.5 [False, True]    main: 1.0 [True, True]

Out of scope

• terminus_env has the same kernel path; Update Terminus env harness and sandbox backends #729 rewrites its e2b_sandbox.py, so I left it alone.
• The submit_paths design from Verify sandbox environments outside the agent's sandbox #1232.